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Abstract — This paper analyzes general spatially-coupled (SC) 
vector systems with multi-dimensional coupling. A continuum ap- 
proximation is used to derive potential functions that characterize 
the performance of the SC systems. For one-dimensional and two- 
dimensional coupling, it is shown that, if the boundary of the SC 
systems is fixed to the unique stable solution that minimizes the 
potential over all stationary solutions, the systems can approach 
the optimal performance as the number of coupled systems tends 
to infinity. Furthermore, a physical intuition is presented for this 
result on the basis of the conservation law of mechanical energy 
in Newton mechanics. 

I. Introduction 

Kudekar et al. fl] proved that spatial coupling can improve 
the belief -propagation (BP) performance of low-density parity- 
check (LDPC) codes up to the maximum-a-posteriori (MAP) 
performance. This phenomenon, called threshold saturation, 
has been observed in many other spatially-coupled (SC) 
systems, such as the MacKay-Neal and Hsu-Anastasopoulos 
codes [2], code-division multiple-access (CDMA) 0-0, 
compressed sensing [6|, [7|, and physical models [8|. Thus, 
threshold saturation via spatial coupling is believed to be a 
universal phenomenon. 

In order to prove the universality of threshold saturation, 
theoretical analyses have been performed for general SC 
systems with one-dimensional coupling [9 1-| 1 1 1. The method- 
ologies are classified into those based on potential func- 
tions 1 9 1, 1 10 1 and on extrinsic information transfer (EXIT) 
functions ifTTl . Potential functions were also used for the 
analysis of threshold saturation in J7], 0. The potential- 
based methodology has the advantage that the analysis of 
the BP performance is simplified. Yedla et al. [10] defined 
a potential function to specify the BP performance for general 
SC systems. However, they presented no derivation of the 
potential function. The main purpose of this paper is to present 
a systematic derivation of the potential function. 

The derivation is based on the continuum approximation 
used in 0, J9). The previous analysis for SC scalar systems 
with one-dimensional coupling is generalized to the case of 
SC vector systems with multi-dimensional coupling. Hereafter, 
general SC vector systems with multi-dimensional coupling is 
simply referred to as generalized SC (GSC) systems. 

The main contributions of this paper are summarized as 
follows: (i) The potential function defined in ifTOl is systemat- 



ically derived via the continuum approximation, (ii) Intuitive 
insights for the threshold-saturation phenomenon are presented 
on the basis of classical mechanics, (iii) Two-dimensional 
coupling is shown to provide the same improvement of the 
BP performance as one-dimensional coupling. 



II. System Model 



A. Notation 

For integers i and j (> i), [i : j] denotes the set {i,i + 
1, . . . ,j} of lattice points. The symbols 8 a i,, 8 b a , S ab represent 
the Kronecker delta. As defined below, the state of a GSC 
system with X-dimensional coupling is represented by two N- 
dimensional vector fields u(x, t) — {u a (x, t) : a — 1, ... , N} 



and v{x 1 t) = {v a (x,t) : a = 1,...,N} on 



I. A' 



[0,oo). 



Roman alphabet is used for the indices of the elements of the 
vector fields, whereas Greek alphabet is for the spatial vector 
x = {x a : a = 1, . . . ,K}. The differential operators d/du a 
and d/dvb are abbreviated to d a and d b , respectively. The 
gradients {d a ■ a = 1, . . . , N} and {d a : a = 1, . . . , N} are 
denoted by V and V, respectively. One the other hand, d/dx 
represents the gradient {d/dx a : a = 1, . . . , K} for the spatial 
variables. Furthermore, the Einstein summation convention is 
used: When an index appears twice in a single term, the 
summation is taken over all values of the index. For example, 

u a v a = }_^ a=1 U a V a . 

B. Uncoupled System 

For two scalar fields F and G on R N , let V C R N and 
V C denote the images of the gradients VF and VG, 
respectively. The dimension N corresponds to the number of 
parameters required for describing asymptotic performance of 
the BP algorithm for a system with no coupling. We assume 
that asymptotic performance is characterized by the coupled 
density-evolution (DE) equations with respect to u(t) = 
{u a (t) : a = 1, . . . , N} and v(t) = {v a (t) : a = 1, . . . , N}, 



u{t + l) = VF(v(t)) e v, 
v(t) = VG(u(i)) € V, 



(1) 
(2) 



where the time index t corresponds to the number of iterations 
for the BP algorithm. The asymptotic performance in itera- 
tion t is assumed to be characterized by a deterministic scalar 



function P(-) of the state u(t) that starts from an appropriate 
initial state. 

Assumption 1. The two scalar fields F and G are thrice 
continuously differentiable on T> and T>, respectively. Let S x 
S C T> x T> denote the set of all fixed-points (FPs) (u,v) 
for the DE equations (0 and (0. The Hesse matrix f ab (v) = 
d a d F(v) is non-singular for all v £ T>q = T*\S, and the 
quadratic form y a ybf ab (v) is bounded below for all {y a } £ T> 
and v £ T>. On the other hand, the Hesse matrix g ab {u) = 
d a dhG{u) are positive (resp. non-negative) definite for u £ 
V = V\S (resp. u £ T>). 

Examples included in our framework are MN and HA 
codes |2|. Note that Assumption is different from that 
postulated in [ 10]: The positive-definiteness of the Hesse 
matrix g a b{u) is assumed in this paper, whereas the positivity 
of its elements is postulated in ifTOl . 

In order to investigate the convergence property of the state 
u(t), we define two potential functions, one of which was 
originally defined in [ 1 1 . They are equivalent to the so-called 
trial entropy for LDPC codes [10] and to the free energy for 
CDMA [4 1, characterizing the MAP performance. 

Definition 1 (Potential). Let D(u, v) denote the divergence 



D(u, v) = G(u) + F(v) - u a v a . 
The potential function V(u) is defined as 
V(u) = -D(u,VG{u)). 



(3) 



(4) 



On the other hand, the dual potential function V(v) is defined 

by 

V (v) = -D(VF(v),v). (5) 

Assumption 2. The potential functions (0 and (0 are 
bounded below. 

Assumption is a sufficient condition for guaranteeing the 
convergence of the state u(t) toward a FP u* as t —> oo. The 
following proposition implies that the two potential functions 
have the same information about FPs. 

Proposition 1. The 2-tuple (u*,v*) is a FP of the DE 

equations (0 and (0 if and only if u* (resp. v*j is a sta- 
tionary solution of the potential (0 (resp. (0). Furthermore, 
V{u*) = V(v*) at any FP (u*,v*). 

Proof: Calculating the gradient of the potential (|4]i with 
(0 yields 



d a V{u) = g ab (u)(u b - d b F{\7G(u)). 



(6) 



Since the Hesse matrix {g a b(u)} of G is non-singular ex- 
cept for the FPs u* £ S, d a V{u*) = is equivalent to 
u* = VF(v*) with v* = VG(u*). Similarly, we find that 
d a V{v*) = is equivalent to the condition that (it*, v*) is a 
FP. Furthermore, we have 




V{VG(u*)) = -D(VF{VG(u*)),VG(u*)) = V(u*), 



(7) 



State u 



Fig. 1. Shape of the potential postulated in this paper. 

where u* — VF(VG(u*)) has been used. This implies 
V(v*) = V(u*). ' ■ 

Let us investigate the convergence property of the state. We 
use the approximation u(t + 1) — u(t) ~ du/dt for (0 to 
obtain 
dn a 

— w -K - d a F{VG{u))} = -g ab {u)d b V{u), (8) 
dt 

for u £ T>o, where we have used (0. In (0, {g ab (u)} 
denotes the inverse matrix of the Hesse matrix {g a t,(u)}. It is 
straightforward to confirm that the potential (0} is a Lyapunov 
function for the continuous-time system (0. 
dV dn a 

— (u(t)) = d a V— = -d a V(u(t))d b V(u(t))g ab (u(t)), 
dt dt (9) 

which is negative for all u(t) £ T>o, and zero only when 
u(t) is at a stationary solution of the potential (0. Since the 
potential (0 is bounded below, this observation implies that 
the state u(t) for the continuous-time system (0 converges to 
a stationary solution of the potential (0 as t — > oo. 

C. General Spatially-Coupled System 

Let us consider the case of double-well potential shown in 
Fig. The potential V(uq) at the left stable solution uq is 
lower than that at the right stable solution «b- Furthermore, 
the performance P(uq) is assumed to be better than P(uq). 
Note that there is no relationship between P{u) and V(u). 
We hereafter refer to uq and u-q as the good and bad 
solutions, respectively. The state u(t) for the BP algorithm 
should converge to the bad stable solution u-q, since the BP 
algorithm starts from a bad initial state. On the other hand, the 
MAP algorithm may achieve the better performance P{uq), 
which implies the suboptimality of the BP algorithm. Spatial 
coupling is a method for helping the state climb the potential 
barrier and arrive at the good stable solution uq. 

Suppose that a GSC system with X-dimensional coupling 
of size L and width W is characterized by the DE equations, 



for all spatial positions I 6 \—L + 1 : L — l] K , with 
1 



(f(m))r 



{2W+ \) K 



ie[-W:W] K 



(11) 



(12) 



We impose the boundary condition u(l,t) = uq for all 
I G Jj \{— L +1 : L — 1] K . This condition corresponds to 
informing the detector about the true solutions for the systems 
at the boundary in advance. 

The idea of spatial coupling is explained as follows: The 
correct information at the boundary may spread over the 
whole system via coupling, regardless of size L. Since the 
influence of the boundary is negligible as L — > oo, the loss 
due to informing the detector about the true solutions is also 
negligible. 

III. Continuum Approximation 

In order to investigate the convergence property of the DE 
equations ( TTOb and ( fTTT i. we use the continuum approximation 
with respect to the spatial variable x = l/L as L — > oo. 
For notational convenience, the argument (x,t) of functions 
is omitted. 



Lemma 1. Let 



1 

L 2 2iy+1 ^ 

v ' m=—W 



(13) 



As L —> oo, the DE equations (Q and (O are respectively 
approximated by the spatially continuous equations 



K 



■i a (x,t+i) = d a F(v) + — j2(r b (v 



dx a dx a 



'dx a2 

o(L 2 ), (14) 



M K 

Va = da G{u) + -Y J 

a = l 



g a b(u) 



d 2 u b 



dx: 



<>2 



-d a d b d c G(u) 



du b du° 
dx a dx a 



o(L 2 



(15) 



Proof: The proof is based on a straightforward Taylor 
expansion with respect to 1/L, and therefore omitted. ■ 
Note that cross terms with respect to the spatial variables 
vanish because of (m) m = 0. In terms of differential geom- 
etry [12], the variable u may be regarded as the coordinates 
associated with a local coordinate system in a differential man- 
ifold with an affine connection. The coefficient d a dbd c G(u) 
in ( TTBT l corresponds to the coefficient of the affine connection 
for the local coordinate system. The same interpretation holds 
for v and d a d b d c F(v). A direct calculation implies that the 
two manifolds for u and v are fiat, i.e. the torsion and 
curvature tensors are everywhere zero. In order to simplify 
the expressions of ( fl4] i and (fT5T l. we use affine coordinate 



systems for which the coefficients of the affine connections 
vanish everywhere. 

Theorem 1. Consider the affine coordinates (u, v) that is 
defined by the twice continuously differentiable mapping from 
(it, v) eV x Vq onto (u, v) £ V x Z> , 



(u,v) = (VF(«),VG(«)). 



(16) 



Fix (x,t) and suppose that (u,v) 6 T>q x 2?o in a neighbor- 
hood of (x, t). Then, the equations ( 1741 ) and ( 1751 ) reduce to the 
two independent equations with respect to the affine coordi- 
nates v = {v a : a = 1, . . . , N} and u = {u a : a = 1, . . . , N}, 



3> a (v(x,t + l)) - $ a (v(x,t)) 

,-^V(^(i)) + M€ a (v)+o(L 2 ), 
av a 



(17) 



# (tt(as,t+l))-¥ (tt(x,t)) 
----^V(*(u)) + M£ a (u)+o(L 2 ), (18) 



with 



£» = ^(/ («)^, + -_(«)__), (19) 

* t~\ ( f ~^d 2 u b , 1 dg ab du b du c 

«.(«) = Y. ( 9a b (u)^ + _(„)__ ] . (20) 



In the first terms on the right-hand sides (RHSs) of ( 17 7| ) 
ant/ ( I7<SI ), f/ie potential functions are given by © an<7 (O, 
respectively. Furthermore, = {$ a (t}) : a = 1, ...,7V} 

ant/ ty(u) = {^ a {u) : a = 1,...,7V} denote the inverse 
mapping of v = VG(m) and u — VF(v), respectively. 

The first terms on the RHSs of ( Tl7l i and (fT8l are potential 
terms determined by the properties of the corresponding un- 
coupled system, whereas the second terms represent the effect 
of spatial coupling. 

Proof: The change of coordinates ( TToT l transforms the 
second term on the RHS of ( fl5l l into 



Q — 1 V 



(21) 



with 



(9 



C = 5 ao (t t ) 5 ec (M)a a 9 £i a e G( M ) + 5ad («)_ 5 -( M ). (22) 

OVc 

Using a formula obtained by differentiating the identity 

g ad {u)g db {u) = «t (23) 
with respect to v c , we find 



C = 5 db ( M ) 3 ec (M)5 a 5 d a e G(«) 



<9i> c 



(u)g db (u) = 0, 



(24) 

for all u e 2?o, which implies that v is affine coordinates. In 
the derivation of (l24l . we have used the chain rule 

d i^{u) = = d a d d d e G(u)g ec (u). (25) 



du e dv c 



Thus, (f2Tb reduces to the simple expression 



M * 8 2 v a 



2 ^ dx 

a—1 



(26) 



with dT6b - Similarly, (fl4] > is transformed into 



M J-^ 8 2 v a 

u a (x, t + l) = d a F(v) + Y Y,d^2+ ( 27 > 

Q— 1 

with u = {u a : a = 1, . . ., JV}. 

We next reduce d26T > and d2Tb to the single equation ([T7l > for 
■S. The derivation of < fT~8T > is performed in the same manner, 
and therefore omitted. Expanding the first term on the RHS 
of (|27) with (|26> yields 



9«F(„) = d a F(v) + ^f ab (v) o(i 2 ). (28) 

a— 1 

We use the chain rule to calculate the second derivative of 
with respect to x a up to the order 0(1), 



d 



dx 
8 2 



^d a F(v) + 0(L- 2 ) 



8x a 

Substituting <|28} and (|29) with u a = d a F(v) into <|27) yields 

w a (a?, t + 1) - u a (cc, i) = -A a + M€ a {v) + o{L 2 ), (30) 

with ([Ell and A a = u a - 8 a F(v). 

In order to prove (TTTt . we shall show that A a is a con- 
servative field with the potential Let T C £>o denote 
a smooth curve connecting two points to and v. When the 
inverse mapping of v — VG(u) maps the curve T to a curve 
r C T>o, the line integral of A a along the curve f yields 



A a d« Q = / k - a a F(VG( M ))} ffa6 (M)^ 6 

r Jr 

=V r (*(«)) + Const, (31) 

where the potential V(u) is given by ©. Expression (l3TT l 
implies that A a is a conservative force. ■ 

IV. Potential Theory 

We replace the difference on the left-hand side (LHS) of 
(fTTI l by the differentiation, with ( fl6l l, to obtain the partial 
differential equation (PDE) 

8v (8 1 

= g ab (*(v)) |-_y(#(«)) + AfC a (i>)j , (32) 

with ( fT9] i, where we have ignored the o(L 2 ) term. This 
replacement might be justified except for initial iterations, 
since the state for large L changes quite slowly as t increases. 
The boundary condition v(x,t) = vq = VG(«g) is imposed 
on the boundary 8Ck of the if -dimensional cubic Ck 
1 . 1] A c R K , since the boundary condition u(l,t) = uq 



has been imposed for all I E Z K \[-L + 1 : L - l] K . Note 
that the state vq corresponds to the good stable solution of 
the potential ©, i.e. V(*(«g)) = V(u G ). 

In order to investigate the convergence property of the 
PDE ( fTTI l. we define the energy functional of a vector field 
v(x) on Ck 



H(v) 



M 



K 



8v a dv b 



2 ^— ' 8x a 8x a 

a—1 



r(v)^dx. 

(33) 

is represented 



It is straightforward to find that the PDE 

t = -«(*»>ffffl, 04) 

where 6H/Sv a denotes the functional derivative of d33l with 
respect to v a . This expression indicates that the state v(x,t) 
moves in a direction where the energy functional (l3Jt de- 
creases. In fact, a direct calculation implies that the energy 
functional (l33l l is a Lyapunov functional for the dynamical 
system d32l : 



dH 



/ —(i>) — (i,)g ab (*(i>))dx<0, (35) 

JCk ° V a 0V b 

where the equality holds only when SH/5v a = is zero 
for all a. Here, we have used the non-negative definiteness 
of the Hesse matrix {g a b(u}}. Furthermore, the energy func- 
tional ( T33l > is obviously bounded below. These observations 
imply that ( T33l > is a Lyapunov functional. Thus, it is guaranteed 
that the state v(x,t) converges to a stationary solution v(x) 
as t — > oo. Note that the convergence of v(x,t) implies the 
convergence lim^oo u(x, t) — u(x). 

Instead of the stationary solution v(x) for d32l , we shall 
investigate properties of the stationary solution u(x) for cno, 
which is a solution to the boundary-value problem 



M£ a {u) = ^(*(«)), 



(36) 



In 



with the boundary condition u(x) = uq for all x £ 8Ck 
(|36| i. we have ignored the o(L 2 ) term. 

In order to obtain an insight based on classical mechanics, 
we assume g a b = 5 a b and K = 1. For notational convenience, 
the superscript 1 is dropped from x 1 for the K = 1 case. 
The differential equation d36l > with d20l > can be regarded as 
the Newton equation of motion: The state u(x) is regarded as 
the position in V of a free particle with vanishing mass M at 
time x, moving subject to the inverted potential —V(i&(u)). 
Note that i is a temporal variable in this interpretation, 
whereas it has been introduced as the spatial variable for the 
SC system. The uniform solution u(x) = uq corresponds to 
the situation under which the particle continues to stay at the 
unstable solution uq of the inverted potential — V(^(u)). On 
the other hand, non-uniform solutions to the situation under 
which the particle at the unstable solution uq at time x = — 1 
moves somewhere, and comes back to «g at time x = 1. 
Vanishing mass M implies that the velocity of the particle 
is infinitely quick. The conservation of mechanical energy 



implies that the latter situation never occurs if uq is the unique 
stable solution that minimizes the potential over all stationary 
solutions. We follow this intuition to prove the following 
theorem for any g a i, and K = 1,2. 

Theorem 2. Let K < 2. the boundary is fixed to the 
unique stable solution that minimizes the dual potential (0 
over all stationary solutions, the uniform solution u{x) = Uq 
is the unique solution to the boundary-value problem ( 1561 ) with 
u(x) = uq for all x £ 8Ck as M — > 0. 

We conjecture that it is possible to prove the same conclu- 
sion for any K. Theorem [2] implies that, if uq satisfies the 
assumption in Theorem [2] asymptotic performance of the BP 
algorithm for the GSC system converges to better performance 
P(uq) for all positions, whereas it for the uncoupled system 
does to worse performance. 

Proof: Let us define the energy tensor Tg(u) as 

mu)=M8^— —g^Q-wU,- ,(37) 



where the dual Lagrangian £ is given by 

M A du a du b 
dx a dx 



t ! u. - | = *(*(«)) + yE 7^7^9a b (u). (38) 



„ du 
dx 

Since the dual Lagrangian ( 08} is invariant under the transla- 
tion of x, we use Noether's theorem to find the conservation 
law 

Tf(u)=-A$(x), (39) 

where Ag(x) is a divergence-free tensor independent of u, 
i.e. dAp/dx a — 0. It is straightforward to confirm the 
conservation law d39l by direct calculation. 

First, assume K = 1. The conservation law (|39l reduces to 



Mdu a du b ~, T ,.,, , 



(40) 



with a constant A, which corresponds to the conservation of 
mechanical energy. It is straightforward to find A = Vg = 
V(^(ug)) as M — > 0, by using the non-negative definiteness 
of g^, the boundary condition u(±l,t) = uq, and the fact 
that du/dx must be infinite for all u ^ S as M — > 0. 

Let us prove u(0) = uq- Since the differential equation (l36l l 
is invariant under the time reversal x = —x, the stationary 
solution has the symmetry u{— x) = u{x). Thus, the differ- 
entiability of u for x implies du(0)/dx = 0. Evaluating ( f40l > 
at the origin x = 0, we find that u(0) — uq must hold from 
the uniqueness of the stable solution Uq. 

The boundary-value problem on the interval [—1,1] has 
been decomposed into two small problems on [—1,0] and 
[0, 1]. From the time-reversal symmetry of (136V the derivative 
of the stationary solution for x is zero at the middle point of 
each interval. This observation and (|40T > with A = Vg imply 
that u(x) must be equal to uq at the middle points. Repeating 
this argument, we find that u(x) = uq must hold at countably 
infinite points. The continuity of the stationary solution implies 
that u{x) = uq for all x G [—1, 1] is the unique solution. 



Next, assume K = 2. The proof is a natural generalization 
of the K = 1 case. We focus on the line segment L\ = 
{(x\x 2 ) : -1 < x 1 < l,x 2 = 0} C [-1,1] 2 of the x 1 - 
axis. Since the differential equation d36l l is invariant under 
the reversal x 2 = —x 2 , we find the symmetry ^(x 1 , —x 2 ) = 
u^x 1 , x 2 ), which implies du/dx 2 = on the line segment C\. 
Thus, the energy tensor (l37t is diagonal on the line segment 
L\ for K = 2, so that A^ is also diagonal on L\ . This implies 
that dAf Jdx a — reduces to dA\/dx 1 — 0. In summary, the 
(1,1) element of (l37T i has the following conservation law 

Mdu a du b ... T> , T ,.,, ,! 

Td^d^ 9ab{u) = v{ * {u)) ~ Al (41) 

on the line segment C%, with a constant A\. 

We find that u(x) = uq must hold on the line segment 
Ci by repeating the argument in the proof for K = 1. 
Similarly, we obtain u(x) — Mq on another line segment 
C 2 = {(x 1 ,^ 2 ) : x 1 = 0,-1 < x 2 < 1} C [-1,1] 2 . Thus, 
the boundary-value problem on the square [— 1, l] 2 has been 
decomposed into four small problems. Repeating the argument 
above implies that u(x) = uq must hold on the square 
[—1, l] 2 , since ii(x) is continuous. ■ 
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